PolarFormer: Multi-Camera 3D Object Detection with Polar Transformer

3D object detection in autonomous driving aims to reason “what” and “where” the objects of interest present a world. Following conventional wisdom previous 2D detection, existing methods often adopt canonical Cartesian coordinate system with perpendicular axis. However, we conjugate that this does not fit nature ego car’s perspective, as each onboard camera perceives world shape wedge intrinsic imaging geometry radical (non perpendicular) Hence, paper advocate exploitation Polar propose new Transformer (PolarFormer) for more accurate bird’s-eye-view (BEV) taking input only multi-camera images. Specifically, design cross-attention based head without restriction structure deal irregular grids. For tackling unconstrained scale variations along Polar’s distance dimension, further introduce multi-scale representation learning strategy. As result, our model can make best use rasterized via attending corresponding image observation sequence-to-sequence fashion subject geometric constraints. Thorough experiments on nuScenes dataset demonstrate PolarFormer outperforms significantly state-of-the-art alternatives.